The present invention relates to head-up displays and, in particular, it concerns a head-up display spatially aligned with the real world for a user viewing a scene through a window.
In the field of military aeronautics, it is known to provide a pilot with a head-up display (HUD). A HUD is a transparent display through which the pilot can see the real world, but which can at the same time present to him additional information, visually superimposed on the real scene. The HUD may be a cockpit-mounted device, in which case it typically covers a limited forward-looking field of view, or may be helmet mounted, in which case it is referred to as a helmet-mounted display (HMD).
In certain cases, a simple HUD system may only display data, such as the state of various aircraft instrumentation, and the location of the data in the field of view is not critical. Of more relevance to the present invention, are spatially aligned HUDs in which the display provides various symbols or other information spatially aligned to appear superimposed over, or otherwise aligned with, a corresponding object or point of interest in the scene viewed by the pilot. This allows the system to prompt the user to look in a specific direction, for example, to identify an incoming target at the limit of the visual range, or to specify what object is being tracked by a weapon system.
Implementation of a spatially aligned HUD is typically highly complex for two reasons. Firstly, complex hardware is used to measure the exact position and pose of the user's head in order to determine the alignment of the HUD with the real world as viewed by the user. Head position is typically measured by tracking the position of the user's helmet within the cockpit, either by a magnetic tracking system or by optical tracking of markers affixed to the helmet. Prior calibration procedures are required in order to determine the position of the user's eyes from the helmet tracking data.
A second reason for the complexity of HUD systems is the use of collimated optics in the display itself. In order for the content of the display to appear in focus while the user is looking at a distant scene, aircraft HUD or HMD systems typically employ an optical arrangement that makes the displayed information and symbols appear to be at infinity, known as collimated optics. This adds significantly to the complexity and weight of the display, and typically limits the available range of viewing angles. Each HUD typically can only be used by a single person at a time, requiring replication of the entire system for each user of the system.
Due to this complexity, and the accompanying costs, use of the HUD has generally been limited to military aircraft and to high-end augmented reality systems. Furthermore, the reliance on a helmet as part of the head tracking system inhibits adoption of HUD technology for various other applications.
Parenthetically, it should be noted that the literature contains many references to head-up displays in the context of automobile instrumentation. However, these HUD systems are typically not spatially aligned displays, as defined above. Techniques such as U.S. Pat. No. 6,750,832 to Peter Kleinschmidt for Information display system for at least one person teach image processing for face detection and gaze direction approximation in a vehicle. This technique teaches deflecting an illumination beam path into the field of view of the observer to display vehicle information, or information requested by the user through a user operated input device. Publication number US 2007-0262971 A1 to Stefan Hahn, Ulm, (DE) for Method and Device for Operating an Optical Display Device teaches a technique for identifying the general direction of a driver's view and varying the display of vehicle information to be compatible with this view. As previously stated, these HUD systems are not spatially aligned displays, as defined above.
There is therefore a need for a spatially aligned head-up display that is a simpler, lower-cost solution that can be used with multiple users. It is desirable that users not be constrained by equipment (such as helmets) and that it is not necessary to require the user to calibrate himself to the system prior to use.